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Abstract 

The aim of this study is to determine through teacher candidates’ thoughts the effects of a portfolio assessment 
implementation on their metacognitive skills and attitudes towards a course on measurement and evaluation. 
Exploratory sequential mixed-methods design is employed within the study. The pretest/posttest control group 
design was used in the qualitative phase of the study and a semi-structured interview form in the quantitative 
phase. The study was conducted with the participation of 42 teacher candidates. Data has been gathered 
using the metacognitive skills scale, an attitude scale on the measurement and evaluation course, and a semi- 
structured interview form. While analyzing quantitative data, assumptions were detected using the SPSS 17.0 
program (multi-normality, extreme values, emission, covariance matrices’ homogeneity, linearity, absence of 
multi-link issues); descriptive statistics and MANCOVA analysis were also been made. The NVivo 8 program 
was used in the analyzing the quantitative data gathered from the semi-structured interview form. Portfolio 
assessments were determined to have positive effects on attendants’ metacognitive skills and attitudes towards 
the course, and the implementation positively affected their attitudes. 
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Within the learning process, requirements like guiding learners and evaluating 
learning tools at different levels put learning into practice, and the concepts of learning 
and evaluating identify with each other instead of competing (Giralt & Varela, 2015; 
Schon, 1987). On this basis and in accordance with the constructivist education 
philosophy, portfolio assessments are recommended for learners’ knowledge, skill, 
and performance to promote their critical and reflective thinking skills (Bahous, 
2008; Conrad, 2008). 

A portfolio is defined as a collection of products that students produce during the 
learning process, and it creates an opportunity for learners, as well as their peers, 
families, and teachers, to observe and evaluate changes over time. Portfolios are also 
thought to be very important in terms of providing direct evidence for quality learning 
media and in-class activities created by teachers (Denney, Grier, & Buchanan, 
2012; Ledoux & McHenry, 2006). Students gather their works systematically and 
methodically in a folder under predetermined criteria. In this way, aside from students’ 
improvements over time, their strengths and weaknesses can also be observed. 

In regard to representing decisions about programs, the teaching process, and 
students, portfolios are significant as a flexible evaluation tool in which different 
students’ products are used as indicators (Davies & Le Mahieu, 2003). Cameron, 
Tate, Macnaughton, and Politano (1998, p. 6) claim that learning takes place only by 
thinking about, problem-solving, constructing, regressing, transforming, reflecting 
on, taking responsibility for, questioning, answering, and implementing knowledge 
about a new situation. Davies and Le Mahieu (2003) also state the main purpose of 
evaluations is to support this. From this perspective, a portfolio is about the concept 
of “evaluating for learning” (Dannefer, 2013; Elango, Jutti, & Lee, 2005; Fung, 
2006). It reveals the negative effect of evaluating students according to their answers 
and telling them their mistakes (Marzano, 2006; Marzano, Pickering, & Pollock, 
2001). Evaluations are highlighted as having as much importance as students’ 
learning processes are in portfolio assessments. Becoming involved in the evaluation 
process requires students to reflect and self-evaluate, as well as to follow learning and 
thinking processes; in other words, students need to use their metacognitive skills. 

The metacognitive concept was coined by Flavell in the 1970s through the results 
of experimental studies on memory processes; it is defined as individual thought 
about one’s cognition by observing one’s own cognitive activities (Flavell, 1979, 
1981). Flavell considers metacognition in the new millennium to be knowledge and 
processes. Knowledge of metacognition includes a comprehensive mental working 
structure in general and the comprehension of one’s own mental working structure in 
particular. On the other hand, metacognition processes include planning, following, 
and arranging thoughts (Flavell, 2000, 2004; Papaleontiou-Louca, 2008). In this 
context, metacognitive skills can be defined as one’s own self-awareness, learning 
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characteristics, and ability to regulate one’s cognitive processes. Hence, Flavell, 
Miller, and Miller (2002) draw attention to metacognition as the key for success in 
different areas like verbal skills, reading, writing, language acquisition, care, memory, 
and social interaction. Senemoglu (2007) also says that metacognitive knowledge is 
rearranged according to metacognitive experiences in the process of arranging an 
individual’s learning activities. 

Metacognition generally means that individuals control their competences and 
skills by observing and making the arrangements they think are necessary. Students 
with metacognitive skills possess the ability to succeed in learning activities and 
responsibilities, and also have a high level of thought when evaluating their own 
comprehension process. In this way, students turn out to be those who make related 
reflections, proving and developing strategies to solve the problems they face in 
this period (Schunk, 2008; Tarricone, 2011). Metacognitive skills are the abilities 
to control and develop cognitive performance. Individuals with metacognitive 
skills often have self-confidence and feelings of self-efficacy, and this situation has 
a positive influence on motivated learning and success (Eisenberg, 2010; Hacker, 
Dunlosky, & Graesser, 2009). 

Reflection and self-arrangement of metacognitive skills are emphasized as they 
are related to critical thinking. At the fore of these skills come thoughts about ideas 
underlying individual’s beliefs and ideas concerning relationships between specific 
concepts and rules (Martinez, 2006). Metacognitive skills include abilities like using 
and choosing proper strategies for learning a subject or condition, evaluating these 
strategies, and making new arrangements by looking at the evaluation results and 
then choosing new strategies. In order for students to gain these skills, they need 
to be placed in learning environments where these skills can be gained effectively. 
Developing metacognitive skills depends on an abundance of previous cognitive 
experiences (Camalahan, 2006; Flavell, 1979; Senemoglu, 2007). Cooper and Sandi- 
Urena (2009) define those with metacognitive skills as ones who can do things 
properly even when they have no experience doing something. 

What is expected from individuals who use metacognitive skills is to know the aim 
of learning, what is expected of them, their position related to the subject, the time 
they need for learning, how to plan as needed for effective learning, how to see the 
positive and negative sides of a plan, how to make the changes a plan needs if the 
majority of its aspects are negative, how to find the strategies needed for use in the 
learning process, and after all of these, how to question whether or not it meets their 
expectations (£ah§kan & Sunbiil, 2011; Senemoglu, 2007). In the problem-solving 
phase, individuals are expected to identify and understand the problem, remember and 
recall prior knowledge, develop a high level of conceptual comprehension, handle the 
problem within a few steps, develop strategies for solving and checking their strategy’s 
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flexibility, and self-evaluate in terms of what is to be done to solve a problem (Haidar 
& Naqabi, 2008; Howard, McGee, Shia, & Hong, 2001). Metacognitive skills have 
the function of both simplifying learning and developing effective problem solving. 
In this respect, it is important to develop media that can improve these skills (Gredler, 
2009; Kapa, 2007). 

In terms of the importance of being active, one characteristic that needs to be 
developed during teacher training for individuals to be effective and life-long learners 
is metacognitive skills. Furthering teacher candidates’ personal and professional 
development requires them to gain efficacy at arranging activities towards improving 
students’ metacognitive skills and at developing the relevant characteristics in their 
pre-service period. Hence, aside from studies in the literature on students of teacher 
candidates, students using web-based training (Baltaci & Akpmar, 2011), strategy 
teaching (Burchard & Swerdzewski, 2009; Chularut & DeBacker, 2004; Pelton, 
2010) and problem-based training (Downing, Kwong, Chan, Lam, & Downing, 
2009; Zhang, Rigdway, & Sachs, 2015) are studies that show the positive effects that 
metacognitive skills have on self-efficacy, success, and self-arrangement (Haryani, 
Prsetya, & Permanasari, 2014; Papaleontiou-Louca, 2003; Metallidou, 2009). 
Portfolio assessment has the feature of supporting high-level thinking skills (Meeus, 
van Petegem, & Meijer, 2008). Teachers and students can clearly see the kind of 
study that needs to be done with portfolios related to concept-learning evaluations 
and students’ learner characteristics and levels. Therefore, students have to think 
about the feedback they receive as a result of their actions and what is expected 
from them. From this point of view, portfolios help students become individuals 
who can use metacognitive knowledge and skills (Clark, 2010). On the other hand, 
portfolio usage is effective in transforming self-regulation into a behavior. In a 
portfolio evaluation process, metacognition is prompted by planning, following, 
and arranging (Zimmerman, 2002). Thus in Baas, Casteljins, Vermulen, Marten, and 
Segers’s (2014) study, metacognition was determined to be stimulated when portfolio 
evaluations are performed. Meyer, Abrami, Wade, Aslan, and Deault (2010) also say 
that portfolio assessment influences the construction of basic metacognitive skills, 
such as students placing themselves in the center of the learning process, setting 
goals, choosing necessary-to-follow strategies, and reflecting on the learning process. 
Students have to think about themselves as both the learning subject and the learner 
in portfolio assessments (McLeod & Vasinda, 2009). In this context, researching the 
usability of portfolio assessment with the aim of developing metacognitive skills 
in teacher education is also seen as important. Portfolio usage in higher education 
and in teacher training has increased since the end of the 1990s. The portfolios used 
in teacher training can be different in content, function, and implementation. All 
the best, complete, and incomplete products teachers put in portfolios can be seen 
gathered. All types of portfolios can be examined under three main topics: efficacy- 
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oriented; action-reflection cycle, and material-focused. Efficacy-oriented portfolios 
create the opportunity to evaluate by integrating with other knowledge and skill- 
measurement tools. Action-reflection cycle portfolios require teacher candidates to 
implement teaching and reflect on their actions. Portfolios focused on media and 
materials require teacher candidates to perform activities where they can show 
personal and learning-related creativity, such as materials, drawings, photographs, 
posters, and created texts (Strijbos, Meeus, & Libotton, 2007). 

Portfolio usage in teacher training is said to have functions like determining 
teacher candidates’ training performance, showing the degree of activity in the 
aims of the teacher-training program (Ledoux & MacHenry, 2006), and creating 
opportunities for teacher candidates to reflect on their training process, output, 
and materials (Zeichner & Wray, 2001). Bloom and Bacon (1995) suggested that 
portfolios develop teacher candidates’ self-reflection and self-evaluation skills and 
simplify integrating decision and problem-solving strategies with teacher candidates’ 
professional skills. Studies emphasizing the importance of portfolio use in teacher 
education underline its effectiveness at developing reflective thinking and learning 
skills (Klenowski, Askew, & Camell, 2006; Lyons, Hyland, & Ryan, 2002); when 
teacher candidates evaluate their own skills, they find opportunities to reflect on their 
in-class activities, knowledge, and skills (Hopfer, 1999). Also, Xu (2004) stated that 
the best aspect of portfolio use in teacher training is that how and when teacher 
candidates do something is known. Denney, Grier, & Buchanan (2012) pointed out 
that portfolios develop professional skills during the pre-service education process; 
they highlighted the importance of expanding portfolio use in teacher-training 
programs. Some findings show that portfolio implementation has a positive effect 
on teacher candidates’ attitudes towards a course (Campbell, Melenyzer, Nettles, & 
Wyman, 2000; Xerri & Campbell, 2015), and that evaluations strengthen motivation 
because it is a stress-free process for learners (Alexiou & Paraskeva, 2010; 2013). 

As no study can be found that emphasizes portfolio assessment’s effects 
on teacher candidates’ metacognitive skills and attitudes towards courses and 
works with both qualitative and quantitative data, this is the basis of our study. 
A measurement and evaluation course has been selected as the main course 
because attention is drawn to the fact that findings in the literature show teacher 
candidates’ attitudes toward measurement and evaluation courses and their sense 
of efficacy to be at low and medium levels (Alkharusi, Kazem, & Al-Musawai, 
2011; Evin-Gencel & Ozba§i, 2013; Ya§ar, 2014). On that note, the aim of 
this study is to determine the effects of portfolio assessment through teacher 
candidates’ thoughts about the implementation in a measurement and evaluation 
course, on their attitudes towards this course, and on their metacognitive skills. 
With this aim, answers to the following questions are sought: 
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Teacher candidates; 

• When checking pretest scores, do metacognitive skills and attitudes about the 
measurement and evaluation course show a meaningful statistical difference for 
the experimental and control groups? 

• What are their opinions about portfolio use in the measurement and evaluation 
course in terms of their attitudes towards the course and their metacognitive 
skills? 


Method 


Research Design 

This study uses the exploratory sequential mixed-method design. In the qualitative 
phase of the study, a pretest-posttest control group match-up design was used; a semi- 
structured interview form was used in the quantitative phase. The mixed research 
method includes gathering and using both qualitative and quantitative methods. In 
this way, a researcher empowers the study by supporting a deficient method with 
another one. The mixed-methods research aims to provide this strength by using 
different methods’ perspectives and their impressions (Creswell, 2006; Johnson & 
Christensen, 2014; Johnson & Onwuegbuzie, 2004). In the first phase of an exploratory 
consecutive mixed-methods design, the process begins with gathering quantitative 
data, and qualitative data is obtained to make better sense of the quantitative data. 
Thus, qualitative data helps explain quantitative data (Creswell, 2013). 

Study Group 

The research study group consists of 42 third-year teacher-candidate students 
studying in a foreign language department at a faculty of education in a Marmara 
area university. The foreign language department was chosen because it had enough 
students to create an experimental group and a control group. Within this study, which 
used a pretest/posttest control group design in its qualitative phase, 15 female and 6 
male teacher candidates participated in the experimental group, and 16 female and 5 
male teacher candidates participated in the control group. 

Data Collection Instruments 

Research data were obtained through the metacognitive skills scale (MS), the 
attitudes toward the measurement and evaluation course scale (ATMEC), and a semi- 
structured interview form. 
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Metacognitive Skills Scale (MS). The MS, developed by Altmdag and 
Senemoglu (2013), is a 5-point Likert-type, one dimensional scale consisting of 30 
items. Cronbach’s alpha-reliability coefficient for the original scale is .94. The one¬ 
dimensional structure of the scale is seen to explain 35.74% of its general variance. 
In the current study, Cronbach’s alpha has been found as .87, which means it provides 
the same reliability as the original. The lowest and the highest scores that can be 
gained from the scale are 30 and 150. 

Attitudes toward the “Measurement and Evaluation” Course Scale (ATMEC). 

This scale, developed by Bryant and Bames (1997) and adapted to Turkish by Ozan 
and Kose (2013), consists of a 5-point Likert structure with three factors. Cronbach’s 
alpha-reliability coefficient for the scale was found as .92 by Ozan and Kose (2013). 
In this study, Cronbach’s alpha reliability coefficient for this scale has been calculated 
as .87. Additionally, the three-dimensional structure of the scale explains 47.7% of 
its general variance. The lowest and highest score that can be achieved from the scale 
are 31 and 155. 

Semi-structured interview form. The semi-structured interview form was used 
with the aim of obtaining the qualitative data of the research. When preparing the form, 
the researcher prepared the questions necessary for finding answers to the research 
questions. These questions were varied with different expressions and then presented 
to specialists in their fields. While taking specialists’ advice, criteria were determined 
as being an expert in education programs, teaching, teaching the measurement and 
evaluation course, and having experience with qualitative studies. With these criteria 
and in light of the specialists’ advice, the semi-structured interview form was put into 
its final structure. 

In the interview form’s final structure were three questions for finding answers 
to the research problems. The researcher read the final state of the form aloud to 
three students who were taking the measurement and evaluation course to test the 
understandability of the questions. The interview process began after this phase. 
Interviews were conducted with voice recordings and the participants’ permission. 
In this sense, interviews were conducted with 14 teacher candidates from the 
experimental group on a volunteer basis. Participants were asked about the portfolio 
assessment method and to be sincere with their answers. The questions directed to the 
participants are listed as follows: 

1. What did you think about the measurement and evaluation course before the 
implementation? What did you think about your own learning process before 
the implementation? 

2. What do you think about the portfolio assessment method? 
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3. What do you now think about the measurement and evaluation course after the 
implementations? What do you think about your own learning process after the 
implementation? 

Data Analysis 

In analyzing the qualitative data, assumptions were detected using the SPSS 17.0 
program (multi-normality, extracting extreme values, homogeneity of covariance 
matrixes, linearity, absence of multi-link problems); MANCOVA and descriptive 
statistics analyses were made. The NVivo 8 program was used in analyzing the 
quantitative data gathered from the semi-structured interview forms. 

Content analysis was performed during quantitative data analysis. In the process 
of content analysis, a three-person committee (the researcher and two specialists) 
were placed to provide reliability. Each of the three committee members separately 
codified each data that had been turned in on the written forms. Miles and 
Huberman’s (1984) formula (Agreement [Agreement + Divergence] x 100) was 
used in the coding process that committee members conducted. The agreement 
percentage among encoders was determined to be 75% for the codes in the first 
question, 80% for the codes in the second question, and 75% for the codes in the 
third question. These percentages show that the research data is reliable. Thus, 
themes were reached by extracting non-concordant codes among the codes 
constructed by the committee from the analysis. To increase the internal validity 
of the research, a detailed literature review was performed before the interviews. 
Additionally, analysis began after receiving confirmation from the participants 
about their written opinions. Ensuring that attendants were sincerely expressing 
their thoughts was attempted during the interview process. To increase external 
validity of the research, what has been performed in the research is explained in 
detail, and learning products and pictures relevant to the portfolio exhibition have 
been included in the appendix with participants’ permissions. Moreover, achieving 
external validity was attempted by allowing explanations on the working group, 
data gathering process, and analysis to occur. 

Processes 

The measurement and evaluation course was taught using the active learning 
method for the experimental and control groups. The semester syllabus was given to 
each group the week before the course began. It explained that evaluations are made 
through a midterm and a final exam. The experimental group received information 
about a three course-hour portfolio assessment, and teacher candidates opted to 
choose the scales for portfolio practice. The experimental process occurred for 10 
weeks after the pretest had been implemented on both groups. The active learning 
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techniques used in the course are the snowball technique, in which individual study 
on a given subject happens first, then in groups of two, four, and eight; the roller 
technique, where discussions are made on a given subject by creating two concentric 
circles; the everyone-is-teacher-here technique, where students write questions about 
the subject and points that need to be explained on small slips of paper which are then 
randomly distributed to the students to await answers; the station technique, which 
requires students to complete other’s incomplete material by creating different activity 
topics; the paper-bag-of-knowledge technique, in which prior knowledge about the 
subject is written on the outside of a paper bag and newly learned information is 
written on small slips of paper; the learning-gallery technique, where what has been 
learned is reviewed and listed, and others add what needs to be added; and the flash- 
card technique, in which green means agree and red means disagree. Courses that use 
active learning techniques perform academic duties such as slogan writing, poetry, 
composition, caricature drawing, educational game development, creative story 
writing, picture-poster drawing, and concept map drawing. Teacher candidates create 
their portfolios by choosing products at the end of these duties under predetermined 
criteria. Data collection instruments were administered as posttests at the end of 
the 10 weeks, after which the products created in relation to the measurement and 
evaluation course and chosen for the experimental group’s portfolios were presented 
as a measurement and evaluation exhibition. The exhibition took place at the faculty 
of education and was open to all instructors and teacher candidates. The exhibition 
invitation cards were made by the teacher candidates, who invited instructors to the 
exhibition by offering instructors an invitation card individually. In the last part of the 
process, interviews were conducted with the experimental group teacher candidates 
using the portfolio semi-structured interview form on portfolio assessments in the 
measurement and evaluation course. 

Data Analysis 

First, the data gathered from the teacher candidates was examined carefully. 
Whether there were any missing or false conveyances within the data was checked 
and any that were found were extracted from the research. Afterwards the data were 
transferred to a computer and examined using SPSS. Points were assigned for missing 
values within the data. Operations were performed that included detecting single and 
multi-directional extreme data values. The Z-value, which is calculated over the total 
scores of the scales to examine unidirectional extreme values, is between +3 and -3. 
Mahalanobis values, which are examined to detect multi-directional extreme values, 
are seen in the square values from Table 5. 
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Findings 

After preparing the data for analysis, it was necessary to question whether the 
assumptions had been met or not in order to perform MANCOVA analysis. Therefore, 
skewness, kurtosis, and histogram graphs of data gathered from scales were first 
examined. As a result of this examination, data was seen to have a normal range. 
Table 1 shows descriptive statistics related to average scores, standard deviations, 
and minimum and maximum skewness and kurtosis values. 


Table 1 

Descriptive Statistics of Data 


Data Collection 

Instruments 

N 

it 

S 

Min. 

Max. 

Skewness 

Kurtosis 

„ Pretest 

MS 

Posttest 

21 

108.29 

12.46 

139 

-0.451 

0.167 

87 

21 

121.93 

8.65 

141 

0.779 

-0.130 

97 

Pretest 

ATMEC 

Posttest 

21 

94.48 

15.14 

119 

-0.333 

-0.420 

59 

21 

121.38 

10.02 

138 

0.098 

-0.663 

95 


Table 1 shows descriptive statistics related to scores gathered from the MS and 
ATMEC scales. Skewness and kurtosis values related to the scales’ values provide 
normality ranges of +/-1. Normal distribution of values, apart from skewness and 
kurtosis coefficients, were examined from the perspective of the Shapiro-Wilk (S-W) 
test. Results of the analysis performed with this aim are shown in Table 2. 


Table 2 

Shapiro-Wilk Test Results for the Values 



S-W Statistic 

Sd 

P 

MS Pretest Total 

.980 

42 

.658 

ATMEC Pretest Total 

.969 

42 

.295 

MS Posttest Total 

.979 

42 

.626 

ATMEC Posttest Total 

.961 

42 

.166 


p > .05. 

In Table 2, scale scores show regular percentages according to the S-W test results 
performed to examine whether or not the MS and ATMEC scales’ pretest and posttest 
scores have a normal rate of distribution,/* > .05, n = 42. Descriptive statistics for the 
MS and ATMEC scales and their data are given in Table 3. 


Table 3 

Pretest-Posttest Descriptive Statistics 



Group 

Mean 

N 

Adjusted Mean 

MS Pretest 

Experimentation 

104.00 

42 

104.00 


Control 

112.57 

42 

112.57 

ATMEC Pretest 

Experimentation 

84.57 

42 

84.57 


Control 

104.38 

42 

104.38 

MS Posttest 

Experimentation 

124.62 

42 

129.01 


Control 

119.24 

42 

114.76 

ATMEC Posttest 

Experimentation 

127.19 

42 

130.06 


Control 

115.57 

42 

112.70 
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Table 3 shows the mean scores and adjusted mean scores of the experimental 
and control groups’ pre- and posttests. The experimental group’s (who practiced 
the implementation) average MS scale score before the implementation was ^ 
= 104.00; after the implementation, it was £, = 124. When examining the control 
group’s average MS scores, their pretest mean score was = 112.57 and posttest 
mean was T, = 119.24. When examined from the perspective of attitudes toward the 
measurement and evaluation course, before the implementation, the ATMEC mean 
for the experimental group who conducted portfolio assessments was = 84.57; after 
the implementation, it was # 2 = 130.06. When examining the control group’s average 
ATMEC scores, their pretest mean score was \ = 104.38; for the posttest, it was £,= 
112.70. 

Apart from the normal distribution of data, there is also a need to prove some 
assumptions in order to be able to conduct MANCOVA analysis. One of these is the 
homogeneity of matrixes’ covariance, whose results were revealed as F _ 288 000) = 
1.414, BoxsM=4AM,p=. 237 , p > .05. For covariance matrixes to be homogeneous, 
p values needed to be insignificant (Tabachnick & Fidell, 2007). When examining p 
values, they are seen to be insignificant, and therefore the matrixes’ covariance is 
homogeneous. 

Another important assumption of MANCOVA analysis is to provide univariate 
homogeneity. Levene test results for the MS posttest total scores, implemented to 
show if there is univariate homogeneity, are F 140) = 2.675, p ~ . 110, p > .05, and for 
the ATMEC posttest total scores, F {] 40) = 3.220, p = .080 ,p> .05. According to these 
results,/? > .05 shows the homogeneity assumption is met. Moreover, the number of 
participants in the experimental and control groups is equal (n = 21). These results 
show it is appropriate to conduct MANCOVA analysis. 

After assuring the MANCOVA assumptions had been met, MANCOVA analysis 
began. The main question of the research is to determine from the results of MANCOVA 
analysis if the posttest scores related to the scales show meaningful difference when 
compared to the pretest scores according to whether or not the evaluation method had 
been implemented for teacher candidates’ metacognitive skills and attitudes towards 
the measurement and evaluation course. The results are presented in Table 4. 


Table 4 

Multi-Variance Analysis Pretest Results 


Variable Source(s) 

Wilk’s Lamda 

Hypothesis df 

Error df 

F 

P 

Partial sum square 

MS Pretest 

.32 

2.000 

37.00 

39.337 

.000* 

.680 

ATMEC Pretest 

.37 

2.000 

37.00 

31.001 

.000* 

.626 


N= 42, */? < .05. 


303 






EDUCATIONAL SCIENCES: THEORY & PRACTICE 


One can see that Wilk’s Lambda values related to both tests are meaningful (p = 0, 
-p < .05). When teacher candidates’ pretest scores from the MS and ATMEC scales 
are checked in Table 4 according to the evaluation method used (classic evaluation 
or portfolio assessment), a meaningful difference between teacher candidates’ scores 
is seen, F (] 40) = 39.337, p < .05 and F {] 40) = 31.001, p < .05). Table 5 shows the 
ANCOVA analysis of adjusted posttest scores according to experimental and control 
groups by retaining the MS and ATMEC scales pretest points. 


Table 5 

ANCOVA Results of Adjusted Posttest Scores according to MS and ATMEC Scales ’Pretest Scores 


Source 

Dependent Variable 

Average of 
Squares 

sd 

Squares total 

F 

P 

Partial 

eta-squared 

Adjusted 

MS Posttest Total 

730.309" 

3 

243.436 

3.952 

.015* 

.238 

Model 

ATMEC Posttest Total 

1,552.184 b 

3 

517.395 

7.675 

.000* 

.377 


MS Posttest Total 

238.878 

1 

238.878 

3.878 

.056* 

.093 

Intercept 

— 







ATMEC Posttest Total 

564.550 

1 

564.550 

8.374 

.006* 

.181 


MS Posttest Total 

601.152 

1 

601.152 

9.760 

.003* 

.204 

Group 

ATMEC Posttest Total 

882.306 

1 

882.306 

13.088 

.001* 

.256 


MS Posttest Total 

2,340.477 

38 

61.592 




Error 

— 






ATMEC Posttest Total 

2,561.721 

38 

67.414 





MS Posttest Total 

627,467.000 

42 





Total 

— 






ATMEC Posttest Total 

622,914.000 

42 





Adjusted 

MS Posttest Total 

3,070.786 

41 





total 

ATMEC Posttest Total 

4,113.905 

41 





a. R 1 = 0.238 (Adjusted i? 2 = 0.178) 

b. R 2 = 0.377 (Adjusted R 2 = 0.328) 


p < .05. 


As seen in Table 5, students’ MS and ATMEC posttest scores show a meaningful 
difference compared to their pretest scores related to the experimental (F (1 = 

9.760,/? < .05) and control groups (F (] Jg) = 13.088,/? < .05). Also, eta-squared values 
related to groups’ scale scores are presented in Table 5. The eta-squared values show 
a significant effect or relationship between variables. This value is important because 
it also defines the degree of significance. By looking at the table above, eta-squared 
values can be seen as .204 and .256, respectively. These results define a mid-level 
effect size for values when comparing the experimental and control groups (Cohen, 
1988; Green & Salkind, 2003). 

Multi-comparisons of posttest scores for the experiment and control groups are 
presented in Table 6. 
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Table 6 

Multi-Comparison of Experimental (ExpGr) and Control (ConGr) Groups’ Posttest Scores 


Dependent 

Variable 

(I) Group 

(J) Group 

Average 

Difference 

(i-j) 

Standard 

P 

95% Confidence Interval of 
Change 3 



Lowest Value 

Highest Value 

MS Posttest 

ExpGr 

ConGr 

14.334’ 

4.588 

.003* 

5.046 

23.623 

Total 

ConGr 

ExpGr 

-14.334* 

4.588 

.003* 

-23.623 

-5.046 

ATMEC 

ExpGr 

ConGr 

17.366’ 

4.800 

.001* 

7.648 

27.083 

Posttest Total 

ConGr 

ExpGr 

-17.366* 

4.800 

.001* 

-27.083 

-7.648 


By looking at Table 6, the change related to MS and ATMEC scores can be seen 
to favor the experimental group. While the groups showed no difference for pretest 
scores, the MS and ATMEC scores for the experimental group who had implemented 
portfolio assessments are considerably higher than the control group’s scores. 

Quantitative findings of the research have revealed that portfolio assessments 
positively affect teacher candidates’ metacognitive skills and attitudes towards the 
measurement and evaluation course. The research’s qualitative findings are presented 
under two headings: participants’ opinions about the measurement and evaluation 
course before and after the implementation. 


Participants’ Opinions of the Course before Implementation 



Figure 1. Pre-implementation NVIVO analysis scheme for emotions that affect 
attitudes towards the course. 

After the interviews with teacher candidates, their pre-intervention opinions on the 
measurement and evaluation course revealed two different themes, prejudgment and 
fear. Looking at the prejudgment theme, one sees the subthemes of difficult course, 
former failures, and I can’t. 
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At the beginning of the school year, former graduates said the measurement and evaluation 
course was hard because of its math. I have no idea about the content because our problem 
was numerical. (Tl) 

Teacher candidates are clearly seen to have had pre-judgmental attitudes toward 
the measurement and evaluation course due to numerical skills and what former 
graduates had said. Thus with regards to group features, some foreign language 
department teacher candidates expressed being pre-judgmental towards courses that 
include math. 

I considered how to prepare for the exam. When I saw math, I thought, “Can I do this?” I 
didn’t have much trouble because it had comment sections. Someone had said there was 
too much math. (A3) 

When examining A3 ’s statement, math activities in the measurement and evaluation 
course were revealed as an important factor affecting students’ prejudgments about 
the course. Negative comments were heard about the course beforehand and had 
affected individuals’ basis of prejudgment. 

I had prejudgments about the measurement and evaluation course because of failing with 
a double F last year. I already thought there were going to be midterm and final 
exams. Midterms and final exams mean failure to me. (A7) 

The example above shows that the teacher candidate had prejudgments about the 
measurement and evaluation course prior to the class. A7 also expressed that the 
course’s classical evaluation process reinforced their negative attitude towards this 
course. On the other hand, individuals’ attitudes toward this course were affected by 
their previous failures. 

I had self-confidence. I also trusted I’d be successful in the pedagogy courses. I had formed 
a prejudgment just because many friends had failed this class. It wasn’t a boring class. It 
could be done. It can be done as long as there are no prejudgments. (A6) 

According to A6’s statement, they had self-confidence but had created a 
prejudgment because of their friends having failed. The participant’s statement that 
course success can be achieved if there are no prejudgments is an important finding. 

I used to say, “Oh, I can’t pass this class!” I would say to myself, “What is this x, what are 
these and those?” (A 10) 

In the statement above, the participant has pre-judgmental opinions toward the 
course and feelings of failure are a result of this prejudgment. 

I told myself I couldn’t do that, but I saw that I could. (A3) 

A3’s statement shows that the participant’s first impression about the course was 
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that they’d fail it. However, the participant is understood to have realized the fear 
was unfounded. 

I am also bad at math; I said “Uh-oh!” I couldn’t bear my fear of math. I told myself I 
couldn’t do the math activities on the exam, either. However, we didn’t get anxious about 
the portfolio assessment process because there was no reason to, and we passed it. We 
evaluated the process. (A11) 

All clearly fears the measurement and evaluation course for its math, and thus 
thinks the evaluation will have poor results. Because portfolio assessments evaluate 
a period rather than just being an exam hour, they are found to affect individuals’ 
attitudes toward the evaluation. 

I already knew we would be asked about this course on the KPSS [Turkish government 
employee entrance exam] before I got to class. I don’t like math. Nevertheless, they said 
my math was insufficient. That’s why I was afraid. When I got to class, I realized I needn’t 
have been afraid of it. (A 12) 

A12 was scared because math was in the measurement and evaluation course 
and they had a negative attitude towards math. However, at the end of the portfolio 
assessment, the participant said their fear was groundless. 

Examining participants’ opinions reveals that the measurement and evaluation 
course had created a negative perspective and fear because of its math-skills activities. 

I had no idea about the measurement and evaluation course at the beginning. I had no friend 
to ask, either. I was afraid of failing when I first attended the course... But there’s nothing 
hard. Now I think I can pass the measurement and evaluation course. I think I can do most 
of the topics once I understand the logic. (A2) 

According to A2’s statement, one deduces that the participant feared failing the 
course before starting. But after the portfolio assessment, they had developed the idea 
that they could pass this course. This statement shows that the participant was fearful 
even before having a negative experience. 

Actually I was afraid of this course because so many students had failed... Your approach 
was so important. (A4) 

Looking at A4’s statement clearly shows they feared failing the measurement and 
evaluation course because of seeing others around them fail. 

I had taken the measurement and evaluation course before from another professor 
and failed. I was afraid of failing. After I began attending the class, I realized I really 
understood the course, and through the evaluations, saw that what I did every week was 
positive because the course was conducted very differently than I was expecting. Moreover, 

I am now thinking about taking undergraduate education in the field of measurement and 
evaluation. (A5) 
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A5’s statement shows a fear of failing gained from previous experience. At the 
end of the implementation, the participant seemed to like the measurement and 
evaluation course, even considered that it deepened their education in this field. That 
the participant also believed they could succeed in their weekly performances is 
thought to be very significant. 

I deserved to fail, and I did fail. This year, I was afraid of not being able to graduate because 
of this course. I realized that I could understand the lesson. I was getting feedback from the 
course. I was able to perform the course’s activities with fondness. (A9) 

A9 shows a fear of failing then begins to believe they can do the class activities. 
This faith influences A9’s success. Factors like being in an activity and providing 
different forms of feedback on the portfolio assessment are also thought to affect this 
situation. 

I was afraid of the course because my roommate had taken this course previously. I told 
myself, “I will definitely fail.” (A 13) 

A13 clearly shows fear of failure because of their friend’s previous failure. 

When looking at the opinions and statements of teacher candidates about the 
measurement and evaluation course before the implementation, they imply having 
prejudgments and fears. Fleading the underlying reasons for this situation are the 
general requirements of complex math skills, previous negative experiences, fear of 
being incapable, and fear of failing the course. 



Figure 2. Post-implementation NVIVO analysis scheme of emotions that affect attitudes towards 
the course. 
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Teacher candidates were asked what they thought about the portfolio assessment 
method to determine their opinions about it. Obtained data are gathered into two 
main themes: positive opinions and negative opinions. Under positive opinions, the 
subthemes formed are socialness, permanent learning, skill discovery, systematical 
study, reinforcement, creativity, and being fun. Also, the subtheme of skill discovery 
is observed to have two subthemes: poetry and painting. Under the other main theme, 
negative opinions, are two subthemes: costly and exhausting. Themes belonging to 
positive opinions follow below. 

Positive Opinions Theme 

I started understanding once we got to the topics. With group activities like this, I had the 
chance to work with people I’d never even said hi to. I liked the group work as well as the 
individual work. (A9) 

A9’s statement shows that group activities conducted within the portfolio 
assessment process gave individuals a chance to know each other better. This shows 
the process is important for in-class interactions. 

We ended up talking about this course on Facebook and the phone. (A2) 

One can deduce from A2’s statement about the portfolio assessment method that 
this implementation affects teacher candidates’ daily lives and provides constant 
communication between individuals on social media and in other areas. 

Personally, the class left permanent marks on me through its portfolio assessment. It was 
also a nice way for me to study for the KPSS... Portfolios answer the question I always ask 
myself, “How can I remember?” In this way, it becomes permanent. (Al) 

By looking at Al’s statement above, one sees that A1 believes what was learned in 
the measurement and evaluation course with portfolio assessments will benefit them 
in the first step of the teaching profession, the KPSS. 

While doing the assignments and activities, when things got tough I’d ask myself, “Why 
are we doing these?” But it was worth it. For example, hmm... While writing a slogan, it 
got stuck in our heads. (A3) 

One sees A3 state that even if it isn’t realized at the time, permanent learning 
reveals itself afterwards. 

I remember what I did in the process better now. This is more pemianent for me. I have 
enjoyed learning by doing something like this. (A4) 

A4 stated that because the portfolio assessment is a process-evaluation method, 
it deepens learning; what was learned in this process becomes more permanent and 
personal. 
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I think portfolios are a very nice idea. We researched step-by-step and it got drilled into our 
brains. I think portfolios are perfect for the measurement and evaluation course. You do 
something with your creations. (A5) 

I especially won’t forget what I wrote with that song’s melody. (A6) 

Of course it was catchy. I learned the topics better, especially those that we studied in detail. (A8) 

According to A5, A6 and A8’s statements, one sees them say the measurement and 
evaluation course based on portfolio assessments is a significant factor that provides 
permanent learning. Implementing different activities, especially those given weekly, 
regularly, and step-by-step, is understood to be important with regard to the constancy 
and continuity of learning. 

I think portfolios were really good. After studying the topic, I tried the assignments. (A9) 

The process is more beneficial than exams, in my opinion, because we cram for exams and 
forget the next day what we had crammed. (All) 

A9 and All’s statements show the importance of activities in this process with 
regard to having students repeat information they’ve gained and providing the 
opportunity to study systematically instead of by memorizing. 

The process was fun. It was really nice to do something colorful and produce different 
things. (A2) 

The slogans and songs were amusing. (A3) 

In light of A2 and A3’s statements on the process of portfolio assessment, the 
process was revealed to be fun and varied, and practicing activities in class makes the 
course entertaining. 

The portfolio implementation seemed so different to me. I mean, last year’s professor was 
so different. We were also learning then, but everything used to stay in the class. Now, the 
topics are strengthened; it is a different kind of study. (A7) 

I saw how important some concepts are, like reliability and validity. I thought these would 
be very useful for me. My course-awareness has increased while having fun, all the while 
strengthening my knowledge. This is a course that should be given to those who will be 
teachers because accurate evaluations are really important. (A 12) 

One can understand from A7 and A12’s statements above that the implementations 
are quite considerable at strengthening knowledge that has been learned. 

During this process, it was good to produce something different. I realized that I had 
composed songs and written poems. This was nice. (A2) 

By the way, within the poem writing activities, I realized that I can write poems. (A4) 
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When the portfolios were revealed, I said, “OK, I can.” I saw that I can draw nice pictures. 

(A 14) 

By looking at the teacher candidates’ statements, this process is clearly important 
for some teacher candidates at discovering new skills. One can say it allows for 
individuals to reveal some skills they already have. 

We revealed our creativeness in the process. We were seeing, thinking, and writing new 
things. (A6) 

According to A6’s statement above on the portfolio assessment process, because 
some new things were revealed, this process can be said to influence one’s creativeness, 
as well. 

Negative Opinions Theme 

Even if it is rare, participants were identified who had stated negative opinions 
about the implementation. 

There were too many assignments in the process; this was exhausting. Also, because I was 
working, I was exhausted. I studied by looking at myself and my deficiencies. Yes it was 
tiring. It’s tiring when there is one each week. (Al) 

The process was exhausting but it wasn’t hard. (A2) 

By looking at teacher candidates’ opinions about the process, one confronts the 
theme of exhausting. According to Al and A2’s above statements, the process was 
revealed to be labeled as exhausting due to reasons like weekly assignments. 

I think the process was a little costly Because there’s always a weekly activity and I had to 
buy things like poster boards, etc. (A2) 

But it was too costly. The poster boards I bought were expensive. (All) 

When looking at negative opinions on the portfolio assessment process, A2 and 
All’s statements reveal that they had found the preparation period of the activities 
to be expensive. 

A general review of the negative opinions on the portfolio assessment process 
gathers them under the themes of exhausting and costly. The reason for calling the 
process exhausting is that the weekly activities and assignments were perceived as 
exhausting by some individuals. Also, the process was found to be costly because 
individuals had to buy things for activities in order to try and enrich the visual 
presentation and give importance to stationary things. 
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Discussion 

Traditional evaluation methods that measure what is remembered and not learned 
are inefficient at determining real learning performance (Rust, 2002). The pre-service 
teacher-training process has changed from teacher-centered and conventionally 
based teaching processes to learner-centered and experience-based processes in 
concordance with modern developments in the teaching-learning field. Within this 
process, it is important that evaluations show a change from being product-focused 
to being period- and performance-focused, as well. However, this situation is not as 
common as it should be. 

This study, which researched teacher candidates’perceptions of modern education’s 
portfolio assessment’s usability and compatibility as well as its influence on their 
attitudes toward a measurement and evaluation course and on their metacognitive 
skills, examined quantitative and qualitative data. The findings are thought to 
contribute to the literature. 

Portfolio assessment is revealed to have a significant positive influence on the 
participants’ metacognitive skills. Alexiou and Parakeva (2010; 2013) emphasized 
that portfolio assessments positively affect university students’ self-arranged learning 
skills and that portfolios can be utilized to improve features like academic success and 
self-efficacy. On one form that supports this statement, examining the findings from 
the interviews with the teacher candidates in the qualitative phase of the research 
shows that the participants produced permanent learning and gained systematic study 
behaviors through the portfolio implementations. 

There are studies revealing the positive effects of portfolio assessments on teacher 
candidates’ high-level thinking and decision-making skills, as well as their problem¬ 
solving and research. Jenson (2011) stated that portfolio assessment develops critical 
reflection skills, and Masters (2013) said it improves self-arranged learning skills. 
Popescu-Mitroia, Todorescu, and Greculescu (2015) determined it to positively 
influence one’s capacity for self-evaluation, creativeness, synthesis, and decision¬ 
making skills. To be able to guide their students in the future, teacher candidates need 
to have the information era’s required skills (lifelong learning and high-level thinking 
and metacognitive skills; Evin-Gencel & Giizel-Candan, 2014). These skills can be 
developed through practical processes during pre-service trainings. When examining 
studies in the literature that had reached the same findings as this one, one sees that 
portfolio assessment is an implementation that can be benefitted from in developing 
metacognitive skills and filling the gap between concept and application. Studies can 
be recommended within the process of accrediting education faculties, which are at 
the front line of Turkey’s Council of Higher Education (YOK) as a current subject in 
teacher training, not just on the benefits of portfolios, but also of e-portfolios. 
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In this study, portfolio assessments were determined to affect teacher candidates’ 
attitudes towards the measurement and evaluation course. Alexiou and Paraskeva 
(2010; 2013) stated that portfolios have an active role in developing positive 
attitudes and affective behaviors. Mitroa, Todorescu, and Greculescu (2015) also 
stated that learners develop motivation and positive attitudes, which is a result of 
their experience in a stress-free evaluation process. Also, there are findings that show 
portfolio assessment positively affects teacher candidates’ attitudes towards courses 
and their learning (Campbell et al., 2000; Xerri & Campbell, 2015). In this context, 
the current research findings agree with the literature. This research’s qualitative 
data also supports the related findings, and teacher candidates stated at the end of 
the study that they had left their fears and pre-judgments about the course behind 
and that the learning process was enjoyable. The participants expressed that in-class 
communication increased while preparing the products for exhibition. They studied 
together beautifully. This allowed them to enjoy the measurement and evaluation 
course, which they had previously prejudged and considered difficult. 

When examining the findings, portfolio assessments are immediately seen to be 
a constructive method for providing both individual and professional development, 
as well as for evaluating their skills. It is recommended to use grading tools such 
as considering more than one evaluator’s opinions, determining the correlation 
between portfolio assessment scores and test scores that record identical features, 
graded point-scoring devices (rubrics), and checking lists to eliminate any reliability 
or validity problems that might come up (Meeus, van Petegema, & Engels, 2009; 
Moya & O’Malley, 1994). Portfolios are significant at providing multiple data for 
developing and evaluating teacher candidates’ performances (Deveci, Ersoy, & 
Ersoy, 2006; Wilbom & Winn, 2000). Within the pre-service training process for 
teacher candidates, learning portfolio implementation through experience can give 
them an opportunity to adapt it into their professional lives. 
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